Normed Linear Spaces

Norm

Definition: Let (V,F)(V,F) be a vector space. A norm on VV is a function

:VR0\| \cdot \| : V \rightarrow \mathbb{R} \geq 0

such that for all x,yVx,y \in V and cFc \in F the following axioms hold:

The triplet (V,F,)(V,F,\| \cdot \|) is called a normed linear space.

Remark: Perhaps the most important property of a norm is that it induces a metric on the vector space. Hence we can quantify the distance between two vectors in a vector space. Namely, the distance between two vectors xx and yy V\in V is the norm of the vector xyx-y or yxy-x :(yx): \|(y-x)\|. Since the norm is always positive, the distance is also positive. The distance between two vectors is zero iff the vectors are the same. x=x0x=x-0, the norm of xx is the distance of xx to the origin. With a proper distance definition (norm), one can begin studying the geometry of the space.

Remark: We can define a "sphere" in VV using the norm concept. The sphere is the set of all vectors in VV with a fixed norm. For example, in R2\mathbb{R}^2, the sphere with radius rr is the set of all vectors with norm rr.

S=vVvv0rS = {v \in V | \|v-v_0\| \leq r}


Example: Let V=R2V = \mathbb{R}^2, F=RF = R, and let x=x = [x1x2]\begin{bmatrix} x_1 \\ x_2 \end{bmatrix}

i. x1:=α1+α2\|x\|_1 := |\alpha_1|+|\alpha_2|, is x1\|x\|_1 a norm?

ii. x2:=x12+x22\|x\|_2 := \sqrt{x_1^2+x_2^2}, is x2\|x\|_2 a norm?

iii.

All these norms can be generalized into what is called the pp-norm.

xp=(i=1nxip)1/p\|x\|_p = \left( \sum_{i=1}^n |x_i|^p \right)^{1/p}

where p1p \geq 1 and x=[x1x2xn]x = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}

Remark: Note that limpxp=x=maxixilim_{p \rightarrow \infty} \|x\|_p = \|x\|_{\infty} = max_i |x_i|

Geometric visualization of the pp-norms in R2\mathbb{R}^2 is given below.

Example: Let x12=(α11/2+α21/2)2\|x\|_{\frac{1}{2}} = ({|\alpha_1|^{1/2}+|\alpha_2|^{1/2})}^2 where, x=[α1α2]x=\begin{bmatrix} \alpha_1 \\ \alpha_2 \end{bmatrix} is x12\|x\|_{\frac{1}{2}} a norm ?


Example: Norms on function spaces. V:{f(.)f[0,1]R s.t. 01f(t)pdt<,1p<}V:\{f(.) | f[0,1] \rightarrow \mathbb{R} \textrm{ s.t. } \int_0^1 f(t)^p dt < \infty , 1 \leq p < \infty \}

Matrix Norms

Definition: Let ARm×nA \in \mathbb{R}^{m \times n} be a matrix. A matrix norm is a function that maps matrices to non-negative real numbers. A matrix norm must satisfy the norm axioms. A norm on matrices can be defined as,

A=maxijaij\|A\| = \max_{ij}|a_{ij}|

where aija_{ij} is the element of AA at the ii th row and jj th column.


Example: Let V=Rn×mV = \mathbb{R}^{n \times m} and A=[aij]A = [a_{ij}]

A1=max1jmi=1naij  (absolute sum of rows)\|A\|_1 = \max_{1 \leq j \leq m} \sum_{i=1}^n |a_{ij}| \ \ \text{(absolute sum of rows)}

Another norm definition is Frobenuis norm:

AF=(i=1nj=1maijp)1p\|A\|_F = (\sum_{i=1}^n \sum_{j=1}^m |a_{ij}|^p)^{\frac{1}{p}}
where 1p\text{where } 1 \leq p \leq \infty

Induced Norm

Definition: A:RnRmA: \mathbb{R}^n \rightarrow \mathbb{R}^m be an m×nm \times n matrix. Let Rn\| \cdot \|_{\mathbb{R}^n} be a norm on Rn\mathbb{R}^n and Rm\| \cdot \|_{\mathbb{R}^m} be a norm on Rm\mathbb{R}^m. The norm of AA induced by these norms is defined as,

A:=maxxRnAxRmxRn\|A\| := \max_{x \in \mathbb{R}^n} \frac{\|Ax\|_{\mathbb{R}^m}}{\|x\|_{\mathbb{R}^n}}

Remark: The induced matrix norm is defined in terms of vector norms. An equivalent definition is given below:

A:=maxxRn=1AxRm\|A\| := \max_{\|x\|_{\mathbb{R}^n} = 1} \|Ax\|_{\mathbb{R}^m}

Remark: The induced norm of a matrix is the maximum amplification of the norm of a vector under the action of the matrix. In other words, the induced norm of a matrix is the maximum amount by which the matrix can stretch a vector.

A nice visualization of the induced norm is given below.


#EE501 - Linear Systems Theory at METU